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Background / Context: 

Description of prior research and its intellectual context. 

Confronted with many challenges to improving the quality of U.S. public K-12 
education, many policymakers have viewed the study of a second language as a useful but non- 
essential ingredient of a world-class education system. However, others point out that dual- 
language education can be a powerful intervention for closing the achievement gap for English 
language learners (ELLs), and that it enhances outcomes for both EEEs and English native 
speakers (Thomas & Collier, 2003; Collier & Thomas, 2004). At the same time, substantial 
evidence attests to the cognitive benefits of early study of another language, and considerable 
evidence supports the relationship between early learning of another language and success in 
other academic subjects. Though numerous studies have established a positive relationship 
between dual-language immersion (DEI) and student achievement in core subjects like 
mathematics and science, questions about these direct relationships remain. Prior studies, which 
are largely descriptive, leave open the question of selection bias because we do not know 
whether the students who gain access to dual-language programs have other characteristics that 
might contribute to the observed outcome of higher academic achievement. This study, which 
incorporates random assignment, helps to assess the causal link between language immersion 
programs and academic achievement. 

The hypothesized relationship between DEI and student outcomes is demonstrated in the 
logic model in Eigure 1, and is based in part on experimental evidence about the cognitive and 
neural basis of math and science skills, as well as the demonstrated relationships among dual- 
language study, cognitive skills, and academic learning. The research suggests that extensive 
engagement in cognitively demanding tasks is associated with gains in cognitive functioning. Eor 
instance, D’Amico and Guamera (2005) show that math performance increases following 
extensive working memory training, and recent work by McClelland et al. (2008) shows the 
transfer of working memory training to comprehension of written material. Cognitive control 
functions, such as working memory and its attention control components, play an important role 
in solving mathematics problems. Working memory skills are also associated with skills and 
knowledge in mathematics and reading comprehensions (e.g., Gathercole et ah, 2006; Alloway, 
2007). In addition, there is significant evidence that bilingualism, which requires students to 
switch attention rapidly from one representational system to another, is correlated with better 
cognitive control and, therefore, may positively affect learning in a wide range of subject areas 
(e.g., Atkins & Baddeley, 1998; Gathercole et ah, 1999). Another example of such research 
comes from Turnbull et al. (2000), who examined the mathematics performance of native- 
English speakers in Canada who were enrolled in a Erench immersion program. The results 
showed that in grade 3, and in grad 6, students in the Erench immersion program actually 
outperformed students in regular and advanced-level mathematics classes in the English-only 
program. 

The question of whether students in immersion schools achieve high levels of language 
proficiency while performing as well or better than their peers in math, science, and English 
language arts has long been settled . The question of whether it is the immersion experience that 
bestows these advantages, however, has not been definitively answered, and that is the primary 
question this study addresses, using evidence from a random- assignment lottery process. 
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Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The proposed paper addresses three research questions about DLI: 

1. What is the effect of dual-language immersion on student achievement in math, science, 
English language arts, and the partner language, and on student attendance and behavior? 

2. How do the achievement, attendance, and behavioral effects of dual-language immersion 
differ for native English speakers versus native speakers of the partner language? 

3. How does DEI program implementation vary across programs, and does that variation 
illuminate differences in program effectiveness? 

Setting: 

Description of the research location. 

The Study takes place in the Portland Public School district (PPS) in Portland, Oregon, the 
largest school district in the Pacific Northwest. PPS serves approximately 47,000 students in 
grades pre-kindergarten through 12. The district includes 85 regular-education schools, as well as 
alternative schools, charter schools, and education services provided at other locations for 
students with special needs. The study focuses in particular on the district’s 19 dual-language 
immersion schools, which include 14 schools with Spanish programs, 3 with Japanese programs, 
3 with Russian programs, and 2 with Mandarin programs. (Three schools are home to more than 
one immersion language.) 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The Study examines seven cohorts of PPS students who entered kindergarten in the academic 
years 2004-05 through 2010-11. The study includes 3,468 lottery applicants, 1,973 of whom 
were randomly assigned to a language immersion program (the treatment condition), and 1,495 
of whom were assigned to the control condition, defined as business as usual. Economically 
disadvantaged students make up about 29% of the DEI program population, and minorities about 
30%; these groups constitute 45% and 45%, respectively, of the district as a whole. English 
language learners make up about 14% of DEI participants and 14% of the district as a whole. 
Students in the analytic dataset are in grades kindergarten through six. 

Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

Dual-language immersion programs are designed to provide both majority and minority 
language speakers with an academically challenging learning environment. Students enter these 
programs as early as kindergarten and continue through high school, studying the same academic 
content as their peers, but in two languages. Such programs aim to help students become 
bilingual, bi-literate, and academically proficient, with strong cross-cultural communication 
skills. 

In Portland, the Russian program and all but one of the Spanish programs follow a two- 
way model in which up to half of the students speak the partner language — Russian or Spanish — 
as their first language, and the remaining students speak English as their first language. Each 
population, therefore, receives instruction in both their native language and a second language. 
The other PPS language programs (Japanese, Mandarin, and one Spanish program), offer a one- 
way immersion model. They cater primarily to native speakers of English and deliver between 
50% and 90% of the standard PPS curriculum in a language other than English. 
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A typical school day for immersion students in Portland is similar to that of other PPS 
students in that they receive the standard district curriculum in a variety of subjects, are held to 
the same academic standards as other students, and are evaluated according to the same battery 
of state assessments. At least half of their day, however, is spent learning the core curriculum in 
a language other than English. Teachers do not spend that time teaching the partner language per 
se, but instead spend it delivering instruction in math, science, social studies, or other subjects 
using the partner language. This approach is developmentally appropriate for elementary-level 
students, who retain the ability to acquire phonology and syntax naturally and efficiently, as 
babies do. In addition, the basic language used to develop kindergartners’ behaviors (“Sit down 
now,” “Look at the board,” “Line up for recess”) and knowledge (colors, shapes, numbers, 
letters) can be easily comprehended and learned with the help of physical and visual cues. While 
the content is the same for immersion and other PPS students, the linguistic environment in 
which they learn that content is significantly different. 

Research Design: 

Description of the research design. 

Our Study capitalizes on Portland Public School’s stratified lottery-based assignment 
system to estimate the unbiased effect of language immersion programs on students’ 
achievement and engagement. We estimate this impact within each subgroup strata in both 
elementary and middle schools, using precision-weighted estimates to obtain average effects. We 
take further advantage of the lottery stratification by estimating the differential effect of the 
immersion program on native English speakers versus native speakers of a program’s target 
language. Linally, we examine implementation of the programs using interviews and classroom 
observations at the school sites. 

We define the treatment as participation from kindergarten in a DLI program within PPS. 
The treatment group consists of applicants admitted to language immersion through the district 
lottery, and the control group consists of immersion lottery applicants who did not win entrance 
to an immersion program, so this causal question concerns the intent-to-treat (ITT) effect (see 
Cullen, Jacob, and Levitt, 2006, for a similarly designed study). The control condition in this 
study is business as usual, defined as instruction in a non-immersion classroom in the district. 

Effects are first pooled and then disaggregated by students’ native language status, which 
is possible due to the presence of distinct lottery strata for native speakers of the partner 
languages. To address non-compliance with lottery status, we use the student’s randomly 
assigned status (lottery winner or loser) as an instrumental variable to estimate the average effect 
of attending a DLI program for individuals who comply with their assigned status (i.e., the Local 
Average Treatment Effect) (Angrist & Pischke, 2009). Moreover, we estimate bounds on the 
treatment effect (Engberg et ah, 2010) to address differential attrition patterns in the treatment 
and control groups. 

To examine implementation, we attempt to observe up to four DLI classes in each DLI 
school, as well as up to four non-DLI classes in similar grades and subjects in the same schools. 
We conduct interviews with principals at each DLI school and with a broad range of central 
office administrators whose work affects the DLI programs. In addition, we aggregate 
observation data to the school level to explore instructional practices as a possible mediator of 
program effects. 
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Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Student achievement in mathematics, science, and English language arts are measured 
with the Oregon Assessment of Knowledge and Skills, a state accountability test measured in 
grades 3 through 8. As measures of student engagement , the study examines students’ annual 
attendance rates and annual number of behavioral referrals. 

We have also collected classroom observation data on 87 classrooms, including 58 
immersion classrooms in 18 schools and 29 non-immersion classrooms in 13 of those immersion 
schools. Observation data are collected by trained observers fluent in the partner languages; 
observers used an adapted form of the district’s sheltered instruction classroom protocol. 

In addition, we have collected implementation perception data from interviews with 17 
immersion school principals and 19 central office administrators, as well as focus group data 
with 32 immersion program teachers from a total of 12 schools. 

Findings / Results: 

Description of the main findings with specific details. 

Analysis of the first years of data from a 3-year study are still under way. Preliminary 
descriptive analyses point to the importance of the random assignment strategy in removing 
selection bias. Simple comparisons of dual-language participants to non-participants in the 
district suggest that mathematics achievement is higher among DEI participants than non- 
participants by between 13 and 24 percent of a standard deviation, and the difference is 
statistically significant at the 1 -percent level, with similar observed relationships in science and 
reading. 

However, such naive estimates are highly vulnerable to selection bias. When the sample 
is limited to lottery applicants using an intent-to-treat framework, the difference becomes small 
and non-significant (and slightly negative in some grades) though analyses are still in progress. 
Importantly, we also find higher rates of attrition among lottery applicants who do not win 
immersion slots, by as much as 21 percentage points. Our current efforts employ a bounding 
analysis (Engberg et ah, 2010) to estimate the effect of immersion under various assumptions 
about the attributes of fa mi lies that were lost to attrition. 

To contextualize these findings, we turn to data from principal and teacher interviews. 
Results from first-year interviews regarding implementation suggest that, in most cases, 
principals perceive dual-language immersion teachers to be among the strongest teachers in their 
buildings. Immersion teachers, on the other hand, describe heavy workloads in terms of 
curriculum and assessment development and in terms of the number of unique courses they are 
expected to teach — especially in the middle and high school grades. Analysis of classroom 
observation data remains in progress. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

As we near the end of the first year of a three-year study, the differential attrition rates among 
lottery applicants who do and do not win DEI slots suggests that DEI programs are an important 
draw for keeping local families enrolled in the public school system. Bounding analyses based 
on assumptions about the attributes of families lost to attrition remain under way but will be 
completed prior to the fall 2013 SREE meeting. In the meantime, qualitative data suggest that 
DEI teachers report challenging working conditions, though principals report satisfaction with 
both the stability and quality of their DEI teachers relative to other teachers in their buildings. 
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External Factors 


Figure 1. Immersion Program Logic Model 
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